Fidelity to Best Practices in EPA Implementation: Outcomes Supporting Use of the Core Components Framework From the University of Virginia Entrustable Professional Activity Program.

PROBLEM: The rapid expansion of entrustable professional activity (EPA) assessment programs has led to calls to ensure fidelity in implementation and integrity in meeting the goals of competency-based medical education. Initiated in July 2017, in advance of the articulated core components of EPA implementation, this article describes the structure and outcomes of the University of Virginia (UVA) EPA Program and provides support for the identified essential components. APPROACH: The UVA EPA Program includes workplace assessments by residents/fellows, attending faculty, and master assessors (MAs), experienced clinicians who assess students across disciplines and clinical settings. All assessors participate in formal professional development and provide verbal and written comments to support their supervision ratings. The Entrustment Committee, composed of 12 MAs, uses a shared mental model and aggregates all assessor data to make a high-stakes summative entrustment decision about students' readiness to assume the role of an acting intern. OUTCOMES: Since 2017, over 2,000 assessors have completed 56,969 EPA assessments for 1,479 students. Ninety-four percent of assessments have been done during the clerkship phase. Residents/fellows have completed a mean of 18 assessments, attending faculty a mean of 27, and MAs a mean of 882. Seventy-four percent of observed encounters involved patients with acute concerns with or without a co-morbid condition. Fifty percent of assessments occurred in inpatient and 32% in ambulatory settings. Eighty-seven percent of assessments contained narrative comments with more than 100 characters. NEXT STEPS: Planned next steps will include earlier identification of students who require individualized learning to promote the development of skills related to EPAs, expansion of the remediation program to enable more students to engage in a clinical performance mastery elective, and creation of targeted professional development for assessors to reinforce the tenets of the EPA program.

Teaching pediatric otoscopy skills to the medical student in the clinical setting: preceptor perspectives and practice.

BACKGROUND: Acute otitis media (AOM) is the most frequent indication for antibiotic treatment of children in the United States. Its diagnosis relies on visualization of the tympanic membrane, a clinical skill acquired through a deliberate approach. Instruction in pediatric otoscopy begins in medical school. Medical students receive their primary experience with pediatric otoscopy during the required pediatric clerkship, traditionally relying on an immersion, apprentice-type learning model. A better understanding of their preceptors' clinical and teaching practices could lead to improved skill acquisition. This study investigates how pediatric preceptors (PP) and members of the Council on Medical Student Education in Pediatrics (COMSEP) perceive teaching otoscopy. METHODS: A 30-item online survey was administered to a purposeful sample of PP at six institutions in 2017. A comparable 23-item survey was administered to members through the 2018 COMSEP Annual Survey. Only COMSEP members who identified themselves as teaching otoscopy to medical students were asked to complete the otoscopy-related questions on the survey. RESULTS: Survey respondents included 58% of PP (180/310) and 44% (152/348) of COMSEP members. Forty-one percent (62/152) of COMSEP member respondents identified themselves as teaching otoscopy and completed the otoscopy-related questions. The majority agreed that standardized curricula are needed (PP 78%, COMSEP members 97%) and that all graduating medical students should be able to perform pediatric otoscopy (PP 95%, COMSEP members 79%). Most respondents reported usefulness of the American Academy of Pediatrics (AAP) AOM guidelines (PP 95%, COMSEP members 100%). More COMSEP members than PP adhered to the AAP's diagnostic criteria (pediatric preceptors 42%, COMSEP members 93%). The most common barriers to teaching otoscopy were a lack of assistive technology (PP 77%, COMSEP members 56%), presence of cerumen (PP 58%, COMSEP members 60%), time to teach in direct patient care (PP 46%, COMSEP members 48%), and parent anxiety (PP 62%, COMSEP members 54%). CONCLUSIONS: Our study identified systemic and individual practice patterns and barriers to teaching pediatric otoscopy. These results can inform education leaders in supporting and enabling preceptors in their clinical teaching. This approach can be adapted to ensure graduating medical students obtain intended core clinical skills.

Moving Toward Summative Competency Assessment to Individualize the Postclerkship Phase.

In the move toward competency-based medical education, leaders have called for standardization of learning outcomes and individualization of the learning process. Significant progress has been made in establishing defined expectations for the knowledge, skills, attitudes, and behaviors required for successful transition to residency training, but individualization of educational processes to assist learners in reaching these competencies has been predominantly conceptual to date. The traditional time-based structure of medical education has posed a challenge to individualization within the curriculum and has led to more attention on innovations that facilitate transition from medical school to residency. However, a shift of focus to the clerkship-to-postclerkship transition point in the undergraduate curriculum provides an opportunity to determine how longitudinal competency-based assessments can be used to facilitate intentional and individualized structuring of the long-debated fourth year.This Perspective demonstrates how 2 institutions-the University of Virginia School of Medicine and the University of Michigan Medical School-are using competency assessments and applying standardized outcomes in decisions about individualization of the postclerkship learning process. One institution assesses Core Entrustable Professional Activities for Entering Residency, whereas the other has incorporated Accreditation Council for Graduate Medical Education core competencies and student career interests to determine degrees of flexibility in the postclerkship phase. Individualization in addition to continued assessment of performance presents an opportunity for intentional use of curriculum time to develop each student to be competently prepared for the transition to residency.

Developing Professionalism and Professional Identity Through Unproctored, Flexible Testing.

The medical education community has devoted a great deal of attention to the development of professionalism in trainees within the context of clinical training-particularly regarding trainees' handling of ethical dilemmas related to clinical care. The community, however, knows comparatively less about the development of professional behavior in medical students during the preclerkship years. In medical schools with flexible testing, students take quizzes or examinations in an unproctored setting at a time of their choosing-as long as it falls within a specified window of time. Unproctored, flexible testing offers students early opportunities to develop appropriate professional behavior. In this Perspective, the authors outline different flexible testing models from three institutions-University of Virginia School of Medicine, University of Michigan Medical School, and Icahn School of Medicine at Mount Sinai-all of which offer various levels of testing flexibility in relation to time and location. The authors' experiences with these models suggest that preclinical medical students' early development of professional behavior requires scaffolding by faculty and staff. Scaffolding involves setting clear, specific expectations for students (often through the form of an honor code), as well as active engagement and discussion with learners about the expectations and procedures for self-regulation in the academic environment.

Teaching Pediatric Otoscopy Skills to Pediatric and Emergency Medicine Residents: A Cross-Institutional Study.

OBJECTIVE: To evaluate a pediatric otoscopy curriculum with the use of outcome measures that included assessment of skills with real patients. METHODS: Thirty-three residents in an intervention group from 2 institutions received the curriculum. In the previous year, 21 residents in a nonintervention group did not receive the curriculum. Both groups were evaluated at the beginning and end of their internship years with the use of the same outcome assessments: 1) a written test, 2) an objective standardized clinical examination (OSCE), and 3) direct observation of skills in real patients with the use of a checklist with established validity. RESULTS: The intervention group had a significant increase in percentage reaching minimum passing levels between the beginning and end of the internship year for the written test (12% vs 97%; P < .001), OSCE (0% vs 78%; P < .001), and direct observation (0% vs 75%; P < .001); significant mean percentage gains for the written test (21%; P < .001), OSCE (28%; P < .001), and direct observation (52%; P = .008); and significantly higher (P < .001) mean percentage gains than the nonintervention group on the written test, OSCE, and direct observation. The nonintervention group did not have a significant increase (P = .99) in percentage reaching minimum passing levels, no significant mean percentage gains in the written test (2.7%; P = .30) and direct observation (6.7%; P = .61), and significant regression in OSCE (-5.2%; P = .03). CONCLUSIONS: A pediatric otoscopy curriculum with multimodal outcome assessments was successfully implemented across different specialties at multiple institutions and found to yield gains, including in skills with real patients.

Responsible Milestone-Based Educational Handover With Individualized Learning Plan From Undergraduate to Graduate Pediatric Medical Education.

Communication of medical students' pediatric milestone assessments and individual learning plans from medical schools to pediatric residency directors allows for effective educational handovers promoting the continuum of education. Existing undergraduate medical education assessments can provide meaningful data to determine most pediatric milestone levels.

The impact of learning communities on interpersonal relationships among medical students.

BACKGROUND: Medical students at the University of Virginia (UVA) are mentored and learn within the framework of a four college learning community. Uniquely, these learning communities are used to organize the third-year clerkship rotations. METHODS: Students were surveyed after their first pre-clinical year and after their clerkship year to determine what the effect of the learning community was on their social and educational interpersonal relationships. RESULTS: Students knew a higher percentage of their college mates after completing their third-year clerkships within the framework of the college system. Students chose peers from within the college system for social and educational interpersonal scenarios statistically more often than what would be expected at random. Small group learning environments that were not formed within the framework of the college system at UVA did not have the same effect on interpersonal relationships, indicating that learning communities are uniquely able to provide a context for relationship building. Students felt more positively about the social and educational effects of the college system after the clerkship year, with a corresponding increase in the strength of their interpersonal bonds with their college peers. CONCLUSION: This work is the first to investigate the effects of learning communities on interpersonal relationships among medical students and finds that learning communities positively impact both social and educational medical student bonds.

Standardized Checklist for Otoscopy Performance Evaluation: A Validation Study of a Tool to Assess Pediatric Otoscopy Skills.

INTRODUCTION: Acute otitis media (AOM) is the most frequently diagnosed pediatric illness in the United States and the most common indication for antibiotic prescription. Skill in pediatric otoscopy is essential to correctly identify children with AOM. However, studies have found diagnostic inconsistency and significant overdiagnosis among practitioners. Given the potential public and individual health consequences, there has been a call for improved education regarding the diagnostic certainty of AOM. Yet educational efforts continue to be limited, particularly in regard to competency assessment. The lack of a validated tool to assess otoscopy skill attainment objectively diminishes the instructor's ability to provide useful feedback and direction to the learner. METHODS: We have undertaken an educational intervention with the goal of developing a validated Standardized Checklist for Otoscopy Performance Evaluation (SCOPE), building on key principles of the general pediatric ear exam. The SCOPE was developed with the input of process and content experts with attention to specific domains of validity. RESULTS: Our analysis provides important validity evidence for the SCOPE assessment tool. The instrument was piloted and successfully implemented with medical students and varying levels of residents in pediatrics and emergency medicine over a 5-year period in varied settings: urgent care, large and small pediatric clinics, and the emergency departments at two institutions. It has been used for both instruction and assessment purposes. DISCUSSION: Because the SCOPE can be used in teaching demonstration purposes, in formative and summative assessment settings, and across the continuum of learners, this instrument offers the potential for more educational efforts in the field of assessment in direct patient care. We anticipate that the SCOPE will foster an environment of efficient yet high-yield review and discussion of otoscopy and diagnostic competency.